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Abstract — Multiple applications that execute concurrently 
on heterogeneous platforms compete for CPU and network 
resources. In this paper we analyze the behavior of K non- 
cooperative schedulers using the optimal strategy that maximize 
their efficiency while fairness is ensured at a system level ignoring 
applications characteristics. We limit our study to simple single- 
level master-worker platforms and to the case where each 
scheduler is in charge of a single application consisting of a large 
number of independent tasks. The tasks of a given application all 
have the same computation and communication requirements, 
but these requirements can vary from one application to an- 
other. In this context, we assume that each scheduler aims at 
maximizing its throughput. We give closed-form formula of the 
equilibrium reached by such a system and study its performance. 
We characterize the situations where this Nash equilibrium is 
optimal (in the Pareto sense) and show that even though no 
catastrophic situation (Braess-Iike paradox) can occur, such an 
equilibrium can be arbitrarily bad for any classical performance 
measure. 

I. Introduction 

The recent evolutions in computer networks technology, as 
well as their diversification, yield to a tremendous change in 
the use of these networks: applications and systems can now 
be designed at a much larger scale than before. Large-scale 
distributed platforms (Grid computing platforms, enterprise 
networks, peer-to-peer systems) result from the collaboration 
of many people. Thus, the scaling evolution we are facing is 
not only dealing with the amount of data and the number 
of computers but also with the number of users and the 
diversity of their needs and behaviors. Therefore computation 
and communication resources have to be efficiently and fairly 
shared between users, otherwise users will leave the group 
and join another one. However, the variety of user profiles 
requires resource sharing to be ensured at a system level. We 
claim that even in a perfect system where every application 
competing on a given resource receives the same share and 
where no degradation of resource usage (e.g., packet loss 
or context switching overhead) occurs when a large number 
of applications use a given resource, non-cooperative usage 
of the system leads to important application performance 
degradation and resource wasting. In this context, we make 
the following contributions: 

• We present a simple yet realistic situation where a 
fair and Pareto-optimal system-level sharing fails to 
achieve an efficient application-level sharing. More pre- 
cisely, we study the situation where multiple applica- 

*The authors would like to thank the University of Tsukuba and the Japan 
Society for the Promotion of Science for supporting this work. 



tions consisting of large numbers of independent iden- 
tical tasks execute concurrently on heterogeneous plat- 
forms and compete for CPU and network resources. 
SETI@home [1], the Mersenne prime search [2], Cli- 
matePrediction.NET [3], Einstein@Home [4], processing 
data of the Large Hadron Collider [5] are a few examples 
of such typical applications. As the tasks of a given 
application all have the same computation and commu- 
nication requirements (but these requirements can vary 
for different applications), each scheduler aims at max- 
imizing its throughput. This framework had previously 
been studied in a cooperative centralized framework [6]. 
In the previous context, at any instant, cooperation led 
to a dedication of resources to applications. The system- 
level resource sharing aspect was therefore not present 
and is extensively described in Section |TI] of the present 
paper. 

We characterize in Section ITl-DI the optimal selfish strat- 
egy for each scheduler (i.e. the scheduling strategy that 
will maximize its own throughput in all circumstances 
and adapt to external usage of resources) and pro- 
pose equivalent representations of such non-cooperative 
schedulers competition (see Section Ullt. 
The particularities of these representations enable us to 
characterize the structure of the resulting Nash equilib- 
rium as well as closed-form values of the throughput of 
each application (see Section Ull-Db . 
Using these closed-form formulas, we derive in Sec- 
tion|IV]a necessary and sufficient condition on the system 
parameters (in term of bandwidth, CPU speed, . . . ) for 
the non-cooperative equilibrium to be Pareto-optimal. 
We briefly study in Section HV-CI the well-known "price 
of anarchy" [7]. Unfortunately, this metric does not 
enable one to distinguish Pareto optimal points from non- 
Pareto optimal ones. That is why we propose an alternate 
definition, the "selfishness degradation factor" . 
When studying properties of Nash equilibria, it is im- 
portant to know whether paradoxical situations like the 
ones exhibited by Braess in his seminal work [8] can 
occur. In such situations, the addition of resource (a new 
machine, more bandwidth or more computing power in 
our framework) can result in a simultaneous degradation 
of the performance of all the users. Such situations only 
occur when the equilibrium is not Pareto-optimal, which 
may be the case in this framework. We investigate in 
Section IIV-DI whether such situations can occur in our 
considered scenario and conclude with a negative answer. 



• Last, we show in Section |V1 that even when the non- 
cooperative equilibrium is Pareto-optimal, the throughput 
of each application is far from being monotonous with 
a resource increase. This enables us to prove that this 
equilibrium can be arbitrarily bad for any of the classical 
performance measures (average, maximal, and minimum 
throughput). 

Section |VT] concludes the paper with a discussion of ex- 
tensions of this work and future directions of research. Due 
to space requirements, the proofs of the following theorems 
and propositions will be omitted in this paper The interested 
reader is referred to [9] for detailed proofs. 

II. Platform and Application Models 

A. Platform Model 

Our master-worker platform is made of + 1 processors 
P07 ^"1, ■ • ■ J ^W- ^0 denotes the master processor, which does 
not perform any computation. Each processor P„ is charac- 
terized by its computing power Wn (in Mflop.s^^) and the 
capacity of its connection with the master _B„ (in Mb.s~^). 
Last, we define the communication-to-computation ratio Cn 
of processor P„ as Bn/Wn- This model leads us to the 
following definition: 

Definition 1. We denote by physical-system a triplet 
{N, B, W) where N is the number of machines, and B and 
W the vectors of size N containing the link capacities and 
the computational powers of the machines. 
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Fig. I. Platform model: a single master and workers 

We assume that the platform performs an ideal fair sharing 
of resources among the various requests. More precisely, let 
us denote by Nn^\t) (resp. N^\t)) the number of ongoing 
communication (resp. computation) from Pq to P„ (resp. 
on P„) at time t. The platform ensures that the amount of 
bandwidth received at time t by a communication from Pq to 
Pn is exactly P„/iV„ '{t). Likewise, the amount of processor 
power received at time i by a computation on P„ is exactly 
Wn/N^\t). Therefore, the time T needed to transfer a file 
of size b from Pg to P„ starting at time to is such that 



to+T 



B„ 



t=to Ni^\t) 



■ dt 



Likewise, the time T needed to perform a computation of size 
w on P„ starting at time to is such that 

rto+T 

■ dt = w. 



t=to Ni'^^t) 



Last, we assume that communications to different processors 
do not interfere. This amounts to say that the network card of 
Pq has a sufficiently large bandwidth not to be a bottleneck. 
Therefore, a process running on Pq can communicate with 
as many processors as it wants. This model is called multi- 
port [10], [11] and is reasonable when workers are spread 
over the Internet, are not too numerous and can make use of 
threads to manage communications. 

B. Application Model 

We consider K applications, Ak, I ^ k ^ K. Each 
application is composed of a large set of independent, same- 
size tasks. We can think of each Ak as a bag of tasks, in 
which each task is a file that requires some processing. A 
task of application Ak is called a task of type k. We let Wk 
be the amount of computation (in Mflop) required to process 
a task of type k. Similarly, bk is the size (in Mb) of (the 
file associated to) a task of type k. We assume that the only 
communication required is outwards from the master, i.e. the 
amount of data returned by the worker is negligible. This is 
a common hypothesis [12] as in steady-state, the output-file 
problem can be reduced to an equivalent problem with bigger 
input-files. Last, we define the communication-to-computation 
ratio Ck of tasks of type k as bk/wk- This model leads to the 
following definition: 

Definition 2. We define a user-system a triplet {K, b, w) 
where K is the number of applications, and b and w the 
vectors of size K representing the size and the amount of 
computation associated to the different applications. 

C. Global System 

In the following our K applications run on our N proces- 
sors and compete for network and CPU access: 

Definition 3. A system 5 is a sextuplet {K, b,w,N,B, W), 
with K,b,w,N,B,W defined as for a user-system and a 
physical-system. 

We assume that each application is scheduled by its own 
scheduler. As each application comprises a very large number 
of independent tasks, trying to optimize the makespan is 
known to be vainly tedious [13] especially when resource 
availability varies over time. Maximizing the throughput of 
a single application is however known to be a much more 
relevant metric in our context [6], [14]. More formally, for a 
given infinite schedule we can define donek{t) the number 
of tasks of type k processed in time interval [0,i]. The 
throughput for application k of such a schedule is defined 
as ak = liminff^tx) .^2!2£fc(£), Similarly we can define a„_fc 
the average number of tasks of type k performed per time- 
unit on the processor P„. ak and a„,fc are linked by the 
following linear equation ak ~ Tlm^n.k- The scheduler of 
each application thus aims at maximizing its own throughput, 
i.e. ak- However, as the applications are sharing the same set 
of resources, we have the following general constraint^ 

Computation Vn e |0, N\ : Y^k=i ^n,k ■ Wk ^ W"„ (la) 

'The notation [a, b\ denotes the set of integers comprised between a and 
h, i.e. [a, fe] = Nn [a,b]. 



Communication e fl, Nj : J2f=i'^n,k ■ bk ^ Bn (lb) 
These constraints enable to define upper-bounds on the 
throughput of any schedule. Moreover, a periodic schedule — 
one that begins and ends in exactly the same state — (we refer 
the interested reader to [15] for more details) can be built from 
any valid values for the ak and an,k such that its throughput 
for all applications k is exactly ak = limt_,oo When 
the number of tasks per application is large, this approach 
has the advantage of avoiding the NP-completeness of the 
makespan optimization problem. This further allows to only 
focus on the average steady-state values. 

Remark 1. Consider a system with K applications running 
over N machines. The set of achievable utilities, that is to 
say the set of possible throughput ak is given by 



known in game theory as Nash equilibrium [16], [17]. In the 



3ai,i, ■ ■ • , aN,K 

Vne [l,iVI 
Vne |l,iV] 



Ylk=l '^n,k ■ 1"k ^ Wn 
Zifc=l Oin.k ■ bk ^ Bn 



The utility set is hence convex and compact. 

D. A Non-Cooperative Game 

We first study the situation where only one application 
is scheduled on the platform. This will enable us to simply 
define the scheduling strategy that will be used by each player 
(scheduler) in the more general case where many applications 
are considered. When there is only one application, our 
problem reduces to the following linear program: 

Maximize Ylin=i "^"-i under the constraints 
' (la) ynelQ,N\:an,i-wi^Wn 
(lb) V?iG[l,iVl:a„,i-6is^B„ 
(Ic) Vti, Q!„4 ^ 0. 

We can easily show that the optimal solution to this linear 
program is obtained by setting Vn, Q!„.i = min 
In a practical setting, this amounts to say that the master 
process will saturate each worker by sending it as many tasks 
as possible. On a stable platform Wn and _B„ can easily be 
measured and the a„ I's can thus easily be computed. On 
an unstable one this may be more tricky. However a simple 
acknowledgment mechanism enables the master process to 
ensure that it is not over-flooding the workers, while always 
converging to the optimal throughput. 

In a multiple-applications context, each player (process) 
strives to optimize its own performance measure (considered 
here to be its throughput ak) regardless of the strategies of the 
other players. Hence, in this scenario, each process constantly 
floods the workers while ensuring that all the tasks it sends are 
performed (e.g., using an acknowledgment mechanism). This 
adaptive strategy automatically cope with other schedulers 
usage of resource and selfishly maximize the throughput of 
each applicatior0. As the players constantly adapt to each 
others' actions, they may (or not) reach some equihbrium, 

^We suppose a purely non-cooperative game where no scheduler decides 
to "ally" to any other (i.e. no coalition is formed). 



remaining of this paper, we will denote by a^"^'' 
achieved at such stable states. 



the rates 



E. A simple example 

Consider a system with two computers 1 and 2, with 
parameters Bi ~ 1, Wi ~ 2, B2 = 2, W2 = 1 and two 
applications of parameters hi = 1, wi = 2, &2 = 2 and 
W2 = 1. If the applications were collaborating such that 
application 1 was processed exclusively to computer 1 and 
application 2 in computer 2 (see Figure |2(a)| i, their respective 
throughput would be 



(coop) (coop) 



1. 



Yet, with the non-cooperative approach (see Figure [2(b)] i, one 
can check that they only get a throughput of (the formal proof 
will be given in Theorem [T]l: 
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(a) With cooperation (b) Non-cooperative equilibrium. 

Fig. 2. Non-cooperation can lead to inefficiencies. 

In this example, one can easily check that, at the Nash 
equilibrium, for any worker, there is no resource waste: slave 1 
(resp. slave 2) is communication (resp. computation) saturated 
i.e. equation ([Tbl i (resp. equation ( fTal i) is an equality. However, 
communication-saturation implies computation idle times and 
vice-versa. Yet, when communication and computation idle 
times cohexist in a system, the non-cooperative behavior of 
the applications can lead to important inefficiencies, which 
we investigate further in the remaining of this paper 

III. Mathematical Formulation 

In this section, we mathematically formulate the use of 
resources in the system. As in the multi-port model com- 
munication resources are all independent, we can study each 
worker separately. Hence, in this section, we study the use of 
resources on an arbitrary worker n of the system. 

In a steady state, actions of the players will interfere on 
each resource in (a priori) a non predictable order and the 
resource usage may be arbitrarily complex (see Figure |3(a)| i. 
We hence propose in this section "equivalent" representations 
(in the sense that they conserve the throughput of each 
application on the considered worker) that will enable us to 
conclude this section with a closed-form expression of the 

(an.fe)i«"««- 

First note that for any given subset K. of we can 

define the fraction of time where all players of K- (and only 
them) use a given resource. This enables us to reorganize the 



schedule into an equivalent representation (see Figure |3(b)| l 
with at most 21'^' time intervals (the number of possible 
choices for the subset JC). In this representation, the frac- 
tions of time spent using a given resource (which can be a 
communication link or a processor) are perfectly equal to the 
ones in the original schedule. However such a representation 
is still too complex (2l'^l is a large value). Hence, we 
now explain how to build two more compact "equivalent" 
canonical representations (see Figure [3(c)| and [3(d)) i. 

A. Sequential Canonical Representation 

The first compact form we define is called sequential 
canonical representation (see Figure |3(c)[ ). If the schedulers 
were sending data one after the other on this link, the 
k^^ scheduler would have to communicate during exactly 



Communication 



Computation 



of the time to send the same amount 



'n,fc Bn 

of data as in the original scheduler. This value is called 
sequential communication time ratio. Similarly, we can define 

the sequential computation time ratio --^ 

We hence have the following relation between r^'^j, 

'n,k 



(B.seq) 



(B.seq) _ Ck (W,seq) 



'n,k 



'n,k 



and 



(2) 



We can therefore obtain a canonical schedule (see Figure [3(c)] i 
with at most K + \ intervals whose respective sets of players 
are {!}, {2}, {A}, 0. This communication scheme is 
thus called sequential canonical representation and has the 
same a^"^'' values as the original schedule. However, com- 
munication and computation times have all been decreased as 
each scheduler is now using the network link and the CPU 
exclusively. We will see later that this information loss does 
not matter for multi-port schedulers. 

B. Parallel Canonical Representation 

The second compact form we define is called parallel 
canonical representation (see Figure |3(d)| i. In this scheme, 
resource usage is as conflicting as possible. Let us denote by 



(resp. T^^^""^') the fraction of time spent by player 
k to communicate with P„ (resp. to compute on P„) in such 
is the parallel communication time 



a configuration, r^jf"'^ 



ratio and t 



{W.par) 



is the parallel computation time ratio. We 



can easily prove that such representation is unique (see the 
extended version [9]) and we can therefore obtain a canonical 
schedule (see Figure |3(d)| i with at most K + 1 intervals 
whose respective player sets are {1, . . . , K}, {2, . . . , A'}, . . . , 
{A'}, and 0. This communication scheme is called parallel 
canonical representation and has the same al,"^' values as 
the original schedule. However, communication times have 
all been increased as each scheduler is now interfering with 
as many other schedulers as possible. 

C. Particularities of Multi-port Selfish Schedulers 

The same reasonings can be applied to computation re- 
sources and therefore, for a given worker, both communication 
and computation resources can be put in any of these two 
canonical forms (see Figure |4(a)| i. 





W,par) 



(a) Parallel canonical form of an arbitrary schedule 
Communication Computation 





(b) Parallel canonical schedule for a given processor under the non- 
cooperative assumption. Application 3 (blue) and 4 (red) are communication 
saturated: they receive the same amount of bandwidth. Application 1 (green) 
and 2 (yellow) are computation saturated: they receive the same amount of 
CPU. 

Fig. 4. Parallel canonical schedules 



As we have seen in Section Hl-DI the scheduling algorithm 
used by the players consists in constantly flooding work- 
ers. Hence it is impossible that both T^y^jf""^^ and 



{W.par) 
n.k 



are smaller than 1. A player k is thus said to be either 

==1) or 

1). 



communication-saturated on worker n (t^^)^^"'^^ 



computation-saturated on worker n (t^'^"''' 
Proposition 1. If there is a communication-saturated appli- 
cation then J2^=i '''luc '''^'^^ = 1- Similarly, if there is a 
computation-saturated application then X^aLi '''n 



(W,seq) 



1. 



(nc) 



impUes J^'llbk, ^ a^nM^k^' ^^"'^^ 

^n./ci ^ ^n.k2 ^™ ^n,fci ^ ^n.fea ' 



As two computation-saturated players ki and k2 receive 
the same amount of computation power and compute during 
the same amount of time, we have q;[j"^|wa;j 
Therefore Cfe^ ^ 

^{B,par) ^ ^{B.par) ^ 

reasoning holds for two communication-saturated players as 
well as for a mixture of both. As a consequence, in a 
multi-port setting, players should be first sorted according to 
their Ck to build the canonical schedule. The very particular 
structure of this schedule (see Figure |4(b)[ ) will enable us 
in the following section to give closed-form formula for the 



(ric) 
^n.k 



All these remarks can be summarized in the following 
proposition: 

Proposition 2. Let us consider an equilibrium and denote by 
Bn the set of communication-saturated applications on worker 
n and by yV„ the set of computation-saturated applications 
on worker n. If ci ^ C2 ^ • ■ • ^ ck, then there exists m G 
|0, A"| such that W„ = |l,m| and fi„ = [m + 1, AT]]. We 
have: 

• Sequential representation: Communications: 

s„ 

(B.seq) , , (B.seq) 



(B.seq) 
7i.Tn+l 



(B.seq) 
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(a) Complex arbitrai'y schedule 



(b) Sorted schedule 
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(c) Sequential canonical representa- (d) Parallel canonical representation: 
tion: areas are preserved but using areas are preserved but using times are 
times are minimized maximized 



Fig. 3. Various schedule representations. Each application is associated to a color: Application 1 is green, application 2 is yellow and application 3 is blue. 
The area associated to each application is preserved throughout all transformations. 



Computations: 

-I ^ AW,seq) _ 
^ 'n,l ~ 



n,m+l ■ ■ ■ ^ '''n.K 



{W,seq) 



Parallel representation: Communications: 



{B,paT) ^ 



, (B.par) (B,par) 

^ ' rj rri \ / ^ i 1 



Tt,m+1 



(Bipar) _ , 



Computations: 

1 _ A^'^P'"') _ 



IW.paT) . (W,par) (W,par) 
'n.m ^ 'n,m+l ^ ' ' ' ^ 'n,K 



D. Closed-form Solution of the Equations 

The closed-form solutions of the equilibrium are defined by 
Theorem [1] Its complete proof, as well as the proofs of the 
other propositions and theorems presented in the remaining 
of this paper can be found in the extended version [9]. 

Theorem 1. We assume ci ^ C2 ^ ■ ■ ■ ^ ck ■ Let us denote 
by W„ the set of players that are computation-saturated and 
by Bn the set of players that are communication-saturated on 
a given arbitrary worker n. 

1) If E/c 7f ^ *en W„ = and 



Vfc, a 



Bn 



2) Else, if Efc ^ < then B,, = and 



K.Wk 



3) Else, Bn and W„ are non-empty and there exists an 
integer m £ |1; /\ — 1| such that 



Cr. 



< 



K - m - Efc=,„+i 



< 



Cm+l 
Cn 



Then, we have yV„ = {!,..., ?ti} and Bn = {m 
l,...,K} and 



(tic) _ _B„ 



|W„|-E„ 



^n,k 



a, 



if fc e Bn 



(3) 



Sketch of the proof. If ;B„ = 0, then all applications use the 
CPU of Pn at any instant. Therefore they all receive the 
exact same amount of CPU, i.e. Wn/K. Hence we have 



^n.k - Moreover, Vfc,T^|^''"'^ = l/K and from © 

we nave i ^ l^k^n,k ^ l^kc^^^nM — l^kKcZ' 
Hence yZi. ^ K- The case W„ = is similar. 

Let us now focus on the more interesting case where both 
B„ 7^ and yV„ ^ 0. Using the definition of sequential 
communication and computation times, we have: 



E 



(B,seq) 



pe8„ ''"".P 



(W,seg) 



+ E 
+ E. 



1 

: 1 



(4) 



Two applications from d communicate all the time. There- 
fore they send the exact same amount of data that we denote 



u w; ^ K> ("c) Bn {B,seq) 

by Vfc e Bn,a' 1^'— = t^^ 

Ok 



rj^K Similarly, 



we get VfceW„,ait^=-S""^-'w 

Wk 

relations and from system (HI can be written: 



t!:a^\ From these 



1 



which can be easily solved to get 

Let m such that m e Z5,i and m + 1 G Wn. From (|2]i and 
(O and Proposition 121 we can write: 



Cn 



and — 



(S,seg) 
n,m+l 

n.m 4-1 
(W,seg) 



> 



< 



(B,seg) 
n,m+l 

n,m-f 1 
'n.m 



Z.^fc=l C„ 



^' - "1 - Efc=™+1 7f 



Z^fc=l c„ 



^---Ef=,„+it 



c„ 



which leads to the condition on m. The reciprocity of the 
conditions on the sets relies on the appUcation of the following 
technical result with jk = Cfe/C„. 

Let 7i < • • • < jK be K positive numbers. We have: 

1) If E^c l/7fc < A' then Efe7fc>A- 

2) If Efc 7fc ^ ^ then 1/7^ > K; 

3) If ^klk > K and Efel/7fc > then there exists 
exactly one 7ti G |1, /ST] such that: 

Em , 

7m =^7f — < Im+l- ■ 

Z^/fc— m+1 



Corollary 1. From these equations, we see that there always 
exists exactly one non-cooperative equilibrium. 

IV. Inefficiencies and Paradoxes 

In this section, we study the inefficiencies of the Nash 
equilibria, in the Pareto sense, and their consequences. Let 
us start by recalling the definition of the Pareto optimality. 

Definition 4 (Pareto optimality). Let G be a game with K 
players. Each of them is defined by a set of possible strategies 
Sk and utility functions Uk defined on <Si x • • • x 5x0 
A vector of strategy is said to be Pareto optimal if it 
is impossible to strictly increase the utility of a player 
without strictly decreasing the one of another In other words, 
{si, . . . , Sk) ^ Si X ■ ■ ■ X Sk is Pareto optimal if and only if: 
V(si, . . . ,Sk) e Si X ■ ■■ X Sk, 

3i, u,;(si, . . . , s*k) > Mi(si, . . . , Sk) =^ 

3j,Uj{sl, . . . ,s*k) < Uj{si, . . . ,sk). 

We recall that, in the considered system, the utility func- 
tions are the a^, that is to say, the average number of tasks 
of application k processed per time-unit, while the strategies 
are the scheduling algorithms (i.e. which resources to use and 
when to use them). 

In this section, we comment on the efficiency of the Nash 
equilibrium, in the case of a single worker (Section FlV-Ab . and 
then of multiple workers dlV-Bl i and propose in Section IIV-CI 
a brief study of the well-known "price of anarchy". Unfor- 
tunately, this metric does not enable to distinguish Pareto 
optimal points from non-Pareto optimal ones. That is why we 
also propose an alternate definition, the "selfishness degrada- 
tion factor'. Last, it is known that when Nash equilibria are 
inefficient, some paradoxical phenomenon can occur (see, for 
instance [8]). We hence study in Section ITV-DI the occurrence 
of Braess paradox in this system. 

A. Single Processor 

We can show that when the players (here the applications) 
compete in a non-cooperative way over a single processor, the 
resulting Nash equilibrium is Pareto optimal (see the extended 
version [9] for a detailed proof). 

Proposition 3. On a single-processor system, the allocation 
at the Nash equilibrium is Pareto optimal. 

B. Multi-processors and Inefficiencies 

Interestingly, although the Nash equilibria are Pareto op- 
timal on any single-worker system, we show in this section 
that these equilibria are not always Pareto optimal for a system 
consisting of several processors. 

We first exhibit this phenomenon on a simple example con- 
sisting of two machines and two applications (Section FlV-B.lb . 
We then provide a very simple characterization of the systems 
under which the non-cooperative competition leads to ineffi- 
ciencies (Section llV-B.2b . 

'Note that the utility of a player depends on its own strategy and on the 
strategies of all the other players. 



1) Example of Pareto inefficiency: The Pareto optimality 
is a global notion. Hence, although for each single-processor 
system, the allocation is Pareto optimal, the result may not 
hold for an arbitrary number of machines. This phenomenon 
was illustrated in Section III-EI 

2) Necessary and sufficient condition: We prove in [9] the 
following very simple characterization of the systems under 
which the non-cooperative competition leads to inefficiencies. 

Theorem 2. Consider a system 5* = {K,b,w, N, B,W) as 
defined in Definition [3] Suppose that the applications are not 
all identical, that is to say that there exists fci and k2 such 
that Cfci < Cfc2 . 

Then, the allocation at the Nash equilibrium is Pareto 
inefficient if and only if there exists two workers, namely 
ni and n2 such that >V„j = and = 0- 

C. Measuring Pareto Inefficiency 

We have seen that the Nash equilibrium of the system can 
be Pareto inefficient. A natural question is then "how much 
inefficient is it?". Unfortunately, measuring Pareto inefficiency 
is still an open question. It is hence the focus of this section. 

1 ) Definitions: Papadimitriou [7] introduced the now pop- 
ular measure "price of anarchy" that we will study in this 
section. 

Let us consider an efficiency measure / on the a^. For 
a given system S (i.e. platform parameters along with the 
description of our K applications), we denote by aj,"^' (5), the 
rates achieved on system S by the non-cooperative algorithm. 
For any given metric /, let (^a^if\s)^ be a vector 

of optimal rates on system S for the metric /. We define 
the inefficiency If{S) of the non-cooperative allocation for a 
given metric and a given system as 

f(a[^\s),...,a^i\s)) 

' ~ f[{a^rHs),...,a^r\s)) 

Papadimitriou focuses on the profit metric E defined by 
I](ai, . . . , uk) = J2^=i ^k- The price of anarchy 0s is 
then be defined as the largest inefficiency: 

</)s max/s(^) = max "'^ ^-^'^ ^ L 
^ ^ Ek^t\S) 

2) Studying the Price of Anarchy on a simple example: 
Let us consider the following simple system Sm.k defined 
by = 1, Bi =1, Wi =1, b = and 
w = (-Jj, 1, . . . , 1). It is then easy to compute the following 
allocations (see Figure |5]i: 

• a'--^'^\SM^K) = • ■ • 1 "^) corresponds to the non- 
cooperative allocation; 

• a^'^'> {Sm,k) = {M, 0, . . . , 0) corresponds to the alloca- 
tion optimizing the average throughput; 

. a("""H5M,if) = ( x-i+i/M ^---^ A'^i+i/A/ ) corre- 
sponds to the max-min fair allocation [18]; 

• a^^\SM,K) = (if'"^'---'"^) corresponds to the pro- 
portionally fair allocation which is a particular Nash 
Bargaining Solution [18]. Surprisingly, on this instance. 
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Fig. 5. Utility set and allocations for Sm,k (K = 3,M = 2). 



this allocation also corresponds to the non-cooperative 
one. 

Note that, a^^\ 

definition. One can easily compute 



, and a^^^ are Pareto optimal by 
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The price of anarchy is therefore unbounded. However, the 
fact that the non-cooperative equilibria of such instances 
are Pareto-optimal and have interesting properties of fairness 
(they correspond to a Nash Bargaining Solution [18]) ques- 
tions the relevance of the price of anarchy notion as a Pareto 
efficiency measure. 

Likewise, the inefficiency of the max-min fair allocation is 
equivalent to M for large values of M (as opposed to K for 
the non-cooperative equilibrium). It can hence be unbounded 
even for bounded number of applications and machines. This 
seems even more surprising as such points generally result 
from complex cooperations and are hence Pareto optimal. 
These remarks raise once more the question of the measure 
of Pareto inefficiency. 

3) Selfishness Degradation Factor: The previous problems 
are not specific to the efficiency measure E. The same kind 
of behavior can be exhibited when using the min or the 
product of the throughputs. That is why we think that Pareto 
inefficiency should be measured as the distance to the Pareto 
border and not to a specific point. 

Based on the definition of the Pareto optimality, one can 
define the concept of strict Pareto-superiority. 

Definition 5 (Pareto-superiority). A utility point a is said 
strictly Pareto-superior to a point (3 if for all player k we 
have Uk > h- 

Obviously, a Pareto-optimal point is such that there is no 
achievable point strictly Pareto-superior to it. To quantify the 
degradation of Braess-like Paradoxes (the degree of Paradox), 
Kameda [19] introduced the Pareto-comparison of a and 
/? as g{a,P) = miuk^- Therefore, a is strictly superior 
to (3 iff g{a, (3) > 1. Intuitively g represents the perfor- 
mance degradation between a and f3. Using this definition, 
we propose the following definition of Pareto inefficiency: 
I{S) = max„gc;(s) e(a,a('"=)(S')). Therefore a("=)(5) is 
Pareto inefficient as soon as I{S) > 1 and the larger 
/(a, the more inefficient the Nash equilibrium. 



The selfishness degradation factor can then be defined as 

ak 



max/(S')=max max min 

S S aeU{S) k 



iS) 



A system (e.g., queuing network, transportation network, 
load-balancing, ...) that would be such that the Nash equi- 
libria are always Pareto optimal would have a selfishness 
degradation factor equal to one. The selfishness degradation 
factor may however be unbounded on systems where non- 
cooperative equilibria are particularly inefficient. The rele- 
vance of this definition is corroborated by the fact that e- 
approximations of Pareto-sets defined by Yannakakis and 
Papadimitriou [20] have a degradation factor of 1 + e. It can 
easily be shown that the systems studied in this article have 
a selfishness degradation factor larger than two but the exact 
value remains an open problem. 

D. Braess-like Paradoxes 

When studying properties of Nash equilibria in routing 
systems, Braess exhibited an example in which, by adding 
resource to the system (in his example, a route), the perfor- 
mance of all the users were degraded [8]. We investigate in 
this section whether such situations can occur in our scenario. 

Let us consider a system (called "initial") and a second one 
(referred to as the "augmented" system), derived from the first 
one by adding some quantity of resource. Intuitively, the Nash 
equilibrium aug in the augmented system should be Pareto- 
superior to the one in the initial system ini. We say that a 
Braess paradox happens when ini is strictly Pareto-superior 
to point aug. 

Obviously, every achievable state in the initial system is 
also achievable in the augmented system. Hence if a is an 
achievable point in the initial system and if & is a Pareto 
optimal point is the augmented one, then a cannot be strictly 
Pareto superior to b. Hence Braess paradoxes are conse- 
quences of the Pareto inefficiencies of the Nash equilibria. 

We show that, even though the Nash equilibria may be 
Pareto inefficient, in the considered scenario, Braess para- 
doxes cannot occur 

Theorem 3. In the non-cooperative multi-port scheduling 
problem, Braess like paradoxes cannot occur 

Sketch of the proof. We first need to introduce the defi- 
nition of equivalent subsystem. Consider a system 5* = 
{KjbjW, N, B,W). We define the new subsystem S = 
{K, b, w, N, B, W) by: for each worker n, 

otherwise. 
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Otherwise. 



We now precise why S is said to be an equivalent subsystem 
of S. Consider a system S = [K, b, w, N, B, W) and its Nash 
equilibrium q;'"^^ One can check that: 

i) The system 5 is a subsystem of S, i.e. for all worker n: 

Bn ^ B„ and Wn Wn- 



ii) The Nash equiHbrium a'-"'^-' of the subsystem S verifies 

iii) The Nash equiUbrium S^'^^) of the subsystem S is Pareto- 
optimal. 

The conclusion of the proof relies on the following result: 
Consider two systems S = {K,b,w, N, B,W) and 5' = 
{K,b,w, N, B' ,W') and their respective equivalent subsys- 
tems S = {K, b, w, N, B, W) and S' = (A', 6, w, N, B\W'). 
Suppose Jhat Vtt^S^ ^ B„ and W/^ ^ W„ then Vn, B'„ ^ 
B„ and W;, ^ Wn. ■ 

V. Performance Measures 

In this section we show that unexpected behavior of some 
typical performance measures can occur even for Pareto 
optimal situations. To ensure optimality of the Nash equilib- 
rium, we consider applications running on a single processor 
(Proposition |3]l. 

We recall that the Pareto optimality is a global performance 
measure. Hence, it is possible that, while the resources of 
the system increase (either by the adding of capacity to a 
link or of computational capabilities to a processor), a given 
performance measure decreases while the equilibrium remains 
Pareto optimal. The aim of this section is to illustrate this 
phenomenon on some typical performance measures. 

More precisely, we show the non-monotonicity of the maxi- 
mal throughput, of the minimal throughput and of the average 
throughput. We finally end this section with a numerical 
example where these measures decrease simultaneously with 
the increase of the resource. 

A. Lower Bound on the Maximal Degradation 

In the following, we suppose that only one of the resource 
of the system increases. By symmetry, we suppose that 
the computational capacity (W^i) is constant, while the link 
capacity Bi increases^ 

Let us introduce B = WK/Y.^ ^ and S ^ X^fc Cfe- 
From Theorem[Tl when considering the equations at the Nash 
equilibrium, we can distinguish 3 cases: 

« 2 "saturated" situations that are: 



If B ^ S, then a,^ 



(nc) 



B 
K.bk ' 



i.e. the 



throughput of each application is proportional to B. 



satBn If B ^ B then a 



(nc) 



-, i.e. the 



throughput of each application is constant with respect 
with B. 

• 1 "continuous" situation when B_< B < B 
Obviously, in the "saturated" situations, the throughput 
a^."^-* are increasing or constant and the order between the 
applications is preserved. (I.e. if for B ^ B_ (resp. B ^ B), 



(nc) 



"fci ' ^ "17'' B' ^B (resp. B' ^ B) we have 

(nc) , (nc) s 

To simplify the analysis, we consider the degradation 
obtained when B = B^ compared to the situation where 

"^In the following, we will omit the subscript "1" as only one worker is 
considered in the system. 



B = B. It is hence a lower bound on the actual maximum 
achievable degradation. 

Consider an arbitrary apphcation k. We write ct'l!^'^\^f^^^ 
) the value of its throughput when B = B_ 



(resp. «I. ,fter 



(resp. B = B). Hence, a 



(nc) 



before 



and 



a 



(nc) 

k after 



W 
Kwk 



Remark 2. Note that 



K 



" : before 

iifler 



The lower 



bound on the degradation is hence proportional to and 
is therefore maximal for the application with the smallest 
coefficient Ck- For instance, if Vp 7^ k,Cp ~ K and = 1, 
then ~ K/2. Hence, when the number of applications 
grows to infinity, the degradation of the application having 
the smaller Cfe also grows to infinity. 

We can now easily show that even in a single processor 
system, the maximal (and minimal) throughput can strictly 
decrease with the adding of resource. Note that: 

* if Bn ^ B_ the application k having the highest (resp. 

smallest) throughput ct"^'^ is the one having the smallest 

(resp. highest) value of bk- 
m if B ^ B the application k having the highest (resp. 

smallest) throughput a^"^'' is the one whose Wk is the 

smallest (resp. highest). 

Hence, a lower bound on the maximal degradation of the 

(resp. 



maximal (resp. minimal) throughput is ^ ^,1 
E i/cfc 'max^rr Therefore, for appropriate choices of 
miiife bk and miufc Wk (for all Ck fixed), the degradation can 
be chosen arbitrarily large. 



miiifc bk 



Finally, note that 



E. l/bk 



Hence, 



for some combinations of Wk, bk and c^, the degradation of 
the average throughput can be arbitrarily large (e.g., for w = 
(1, 1/e, . . . , 1/e) and b = (1, . . . , we get a ratio 



of K 



(l + (X-l)e)2 







K). 



B. Numerical Example 

We end this section with an example in which all the perfor- 
mance measures we considered are simultaneously degraded 
when the bandwidth B of the link connecting the master to 
the worker is increased. 

Consider the example represented in Fig. |6] Observe that 
when the bandwidth B is 245/24 ~ 10.208 the three measures 
(namely the higher throughput, the lower throughput and 
the average throughput) have lower values than when the 
bandwidth B is only equal to 560/73 2± 7.671. 

As miufc Wk = W4 and miiife bk = ^4, then the application 
having the higher throughput in both satBn and satWn is 
application 4, and a lower bound of the degradation is 112/73. 

As maxkWk = and max bk ~ 61, then the application 
having the lower throughput is application 3 in satBn and 
application 1 in satWn, and a lower bound of the degradation 
is 84/73. 

Finally, a lower bound of the degradation of the average 
performance is 2466/2263. 




Fig. 6. The three performance measures can simultaneously decrease with 
the resource: b = I 8,5,7,1), w = I 4,5,12,21, K=4, W =10. 



VI. Conclusion 

We have presented a simple yet realistic situation where a 
fair and Pareto-optimal system-level sharing fails to achieve 
an efficient application-level sharing. Even though the system 
achieves a perfect sharing of resources between applications, 
the non-cooperative usage of the system leads to important 
application performance degradation and resource wasting. 
We have proved the existence and uniqueness of the Nash 
equilibrium in our framework and extensively studied its prop- 
erty. Surprisingly, the equilibrium is Pareto-optimal on each 
worker independently. However, it may not be globally Pareto- 
optimal. We have proved that no Braess-like paradoxical 
situations could occur, which is, to the best of our knowledge, 
the first situation where Pareto-inefficient non-cooperative 
equilibrium cannot lead to Braess-like paradox. However, 
some seemingly paradoxical situations can occur Indeed, even 
when the equilibria are Pareto optimal, their performance can 
be arbitrarily bad for any classical performance measure. 

This study led us to the natural question of the inefficency 
measure. After briefly commenting on the notion of "price 
of anarchy", we proposed a new definition, called SDF 
(Selfishness Degradation Factor). 

The key hypothesis for deriving a closed-form descrip- 
tion of the equilibria is the multi-port hypothesis. Under 
this hypothesis, some time information could be lost when 
using equivalent representations, which resulted in simpler 
equations than if a 1-port model had been used (i.e. if the 
master can communicate with only one worker at a given 
instant). Preliminary simulations with this model show that 
Braess-like paradoxes may occur The understanding of such 
phenomena are crucial to large-scale system planing and 
development as there is no way to predict their apparition so 
far. Analytical characterizations of such a framework could 
provide significant insights on the key ingredients necessary 
to the occurrence of Braess-like paradoxes. 

Last, we can conclude from this study that cooperation 
between applications is essential to avoid inefficiencies (even 
for simple applications constituted of a huge number of 
independent identical tasks). As far as the framework of this 
article is concerned, some steps in this direction have been 
given in [6] where some distributed algorithms were proposed 
and compared to an optimal but centralized one. However, in 



their work, there was a single scheduler whose duty was to 
achieve the best throughput for all applications while ensuring 
a max-min fair share. In a fully-decentralized setting, as 
considered in the present article, some form of cooperation 
(e.g., similar to the one proposed by [21] for elastic traffic in 
broadband networks) between different schedulers should be 
designed. 
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